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(54) Method and apparatus for ultrasound imaging of a biopsy needle 

(57) A method and apparatus for enabling a biopsy 
needle to be observed in a three-dimensional ultra- 
sound diagnostic system using an interventional ultra- 
sound system. The apparatus comprises an ultrasound 
transducer, a three-dimensional image-forming section, 
a section for extracting a target object, a location-calcu- 
lating section, a display section, and a controller. The 
method comprises the steps of acquiring a two-dimen- 
sional ultrasound image of a subject; generating a three- 
dimensional volume image based on the two-dimen- 
sional ultrasound image; segmenting a target image, 
which corresponds to a target object within the subject, 
from the three-dimensional volume image; displaying a 
guide line of the biopsy guide on the segmented target 
image; extracting the segmented target image; acquir- 
ing information on a location of the biopsy needle by ref- 
erence to the segmented target image; calculating an 
error based on the information; and displaying a guiding 
status of the biopsy needle based on the calculated er- 
ror. 
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Description 

[0001] The present invention relates to an ultrasound 
diagnostic system, and more particularly, to a method 
and apparatus for observing a biopsy needle and guid- 
ing the same towards a target object within the human 
body in a three-dimensional ultrasound diagnostic sys- 
tem that uses interventional ultrasound system. 
[0002] In the field of interventional ultrasound imag- 
ing, a medical procedure is performed which, for diag- 
nostic or treatment purposes, takes a sample of a target 
object, for example, the tissue of a specific internal or- 
gan of the patient's body by inserting a medical device 
such as a biopsy needle into the patient's body and, at 
the same time, monitoring the movement of such med- 
ical device within the patient's body in real time with the 
use of the ultrasound diagnostic system. Thus, in order 
to ensure reliable diagnosis, it is necessary to display 
an ultrasound image in such manner that a positional 
relationship between the target object and the biopsy 
needle can precisely be identified. 
[0003] However, since a conventional two-dimension- 
al ultrasound diagnostic system displays the biopsy 
needle in a plane defined by the axial and lateral direc- 
tions of the transducer, it is difficult to precisely identify 
the location of the biopsy needle in the displayed ultra- 
sound image when a movement in elevation of the bi- 
opsy needleoccurs. Asimilar problem arises forthetwo- 
dimensional ultrasound diagnostic system which can 
generate a two-dimensional B-mode image of human 
organs and can show the guide line of the biopsy needle 
on the image being displayed by using a geometrically 
adequate biopsy guide. More specifically, since the 
transducer used in the two-dimensional ultrasound di- 
agnostic system has to perform beam-focusing in the 
direction of elevation as well, it is not viable to display 
the movement in elevation of the biopsy needle in a two- 
dimensional image when the biopsy needle is not locat- 
ed in a plane defined by the axial and lateral directions, 
i.e., when the movement of the biopsy needle is to the 
direction of elevation. The difficulty in observing a target 
object in the displayed ultrasound image is attributed 
mainly to the fact that most of the target objects take a 
form of a three-dimensional sphere, while the two-di- 
mensional ultrasound diagnostic system displays only 
a sectional view of the portion where the transducer is 
currently located. Therefore, visualization capabilities of 
a target object to be subject to a tissue inspection are 
inevitably restricted by the inherentcharacteristicsofthe 
two-dimensional ultrasound diagnostic system. 
[0004] In an attempt to circumvent the above prob- 
lems, a three-dimensional ultrasound diagnostic system 
is often used when performing a medical inspection, 
where a biopsy needle is used, due to the fact that a 
positional relationship between the biopsy needle and 
the target object can clearly be observed in a volume 
image provided by the three-dimensional ultrasound di- 
agnostic system. Among the several methods for ob- 



serving the biopsy needle adopted for the three-dimen- 
sional ultrasound diagnostic system, the following two 
methods are known as being most widely used. The first 
is a method wherein the biopsy guide is attached to the 

5 transducer as in a conventional two-dimensional diag- 
nostic system, and the biopsy needle is displayed by us- 
ing known information on a geometrical structure of the 
biopsy needle; and the other is a method called "a free- 
hand style method." In the free-hand style method, the 

10 location of the biopsy needle is discerned by the user's 
estimation or sensation wherein the user holds a three- 
dimensional transducer with one hand as shown in Fig. 
1, and manipulates the biopsy needle with the other 
hand without using the biopsy guide. 

15 [0005] As described above, the three-dimensional ul- 
trasound diagnostic system can also provide a guide 
line as is provided by a two-dimensional ultrasound di- 
agnostic system, in case that the biopsy guide is used. 
However, in a display image showing a volume rendered 

20 with respect to x, y and z axes, the biopsy needle is typ- 
ically displayed on one of the three planes defined by 
the x, y and z axes. Thus, unless the three-dimensional 
ultrasound image is rearranged, it is difficult to precisely 
observe the movement of the biopsy needle since the 

25 biopsy needle is indicated merely as a point in the ren- 
dered image. The free-hand style method cannot dis- 
play a location of the biopsy needle precisely. Further, 
for the free-hand style method, diagnosis is performed 
mainly by estimation or sensation of the user so that the 

30 accuracy of diagnosis depends on the skill of the user. 
[0006] It is accordingly an object of the present inven- 
tion to provide a method and apparatus for accentuating 
a relationship between a biopsy needle and a target ob- 
ject by contouring the biopsy needle in a three-dimen- 

35 sional ultrasound image provided by a three-dimension- 
al ultrasound diagnostic system. 
[0007] It is another object of the present invention to 
provide a method and apparatus for automatically guid- 
ing a biopsy needle into the human body by automati- 

40 cally extracting a target object from a three-dimensional 
ultrasound image, contouring the target object, and ob- 
taining information on locations of and a relationship be- 
tween the extracted target object and the biopsy needle 
being guided. 

45 [0008] In accordance with one aspect of the present 
invention, an apparatus for enabling a biopsy needle to 
be observed in a three-dimensional ultrasound image is 
provided which comprises means for transmitting ultra- 
sound signals towards a target object and receiving 

50 echo signals reflected from the target object; means for 
forming a three-dimensional volume image by receiving 
and combining two-dimensional data from the transmit- 
ting means; means for extracting the target object from 
the three-dimensional volume image; means for esti- 

55 mating a location of the extracted target object and a 
location of the biopsy needle that is inserted into the ex- 
tracted target object; means for displaying the extracted 
target object; and means for calculating an error in lo- 
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cation between the biopsy needle and the extracted tar- 
get object based on the estimation and providing the cal- 
culated error to the estimating means. 
[0009] In accordance with another aspect of the 
present invention, a method of enabling a biopsy needle 
to be observed in a three-dimensional ultrasound image 
is provided which comprises the steps of acquiring a 
two-dimensional ultrasound image of a subject; gener- 
ating a three-dimensional volume image based on the 
two-dimensional ultrasound image; segmenting a target 
image, which corresponds to a target object within the 
subject, from the three-dimensional volume image; dis- 
playing a guide line of the biopsy guide on the segment- 
ed target image; extracting the segmented target image; 
acquiring information on a location of the biopsy needle 
by reference to the segmented target image; calculating 
an error based on the information; and displaying a guid- 
ing status of the biopsy needle based on the calculated 
error. 

[0010] The features, objects, and advantages of the 
present invention may best be understood with refer- 
ence to the following description, taken in conjunction 
with the accompanying drawings. 

Fig. 1 shows a structure of a conventional interven- 
tional ultrasound imaging system. 
Fig. 2 shows a schematic block diagram of an ap- 
paratus for enabling observation of a biopsy needle 
in a three-dimensional ultrasound image according 
to the present invention. 

Figs. 3a and 3b show ultrasound images to explain 
a procedure for extracting a target object from 
three-dimensional ultrasound image data acquired 
in accordance with the present invention. 
Figs. 4a and 4b show ultrasound images to explain 
a procedure for guiding a biopsy needle by extract- 
ing a target object in accordance with the present 
invention. 

Fig. 5 shows a flowchart to explain a procedure for 
observing a biopsy needle in a three-dimensional 
ultrasound image provided in accordance with the 
present invention. 

[001 1 ] Referring to Fig. 2, there is shown a schematic 
block diagram of an apparatus for enabling observation 
of a biopsy needle in athree-dimensional ultrasound im- 
age according to the present invention. As shown in Fig. 
2, a guiding apparatus 100 comprises an ultrasound 
transducer 102, a three-dimensional image-forming 
section 1 04, a section for extracting a target object 1 06, 
and a location-calculating section 1 08, a display section 
110, and a controller 112. Ultrasound transducer 102 
serves to transmit ultrasound signals towards a target 
object and receive echo signals reflected from the target 
object. Either a rotated transducer by motor or an elec- 
tronic transducer with piezo-electric elements arranged 
two dimensionally may be used as ultrasound transduc- 
er 1 20. However, input data format differs depending on 



which transducer is used for ultrasound transducer 1 02. 
To this end, a coordinate system defined by an angle 
and a distance or a three-dimensional rectangular coor- 
dinate system may be chosen to adequately represent 

5 the locations of pixels. Three-dimensional image-form- 
ing section 104 combines continuous two-dimensional 
data inputted from ultrasound transducer 102 to pro- 
duce three-dimensional volume data. 
[0012] Section for extracting a target object 106 ex- 

10 tracts a target object from the three-dimensional volume 
data inputted from three-dimensional image-forming 
section 104. In this case, a boundary of the target object 
may be extracted by using the image processing tech- 
nique called "Virtual Organ Computer Aided Analysis 

15 (VOCAL)" disclosed in Korean Patent No. 1 0-0308230, 
issued August 27, 2001 and in European Patent Appli- 
cation No. EP 1 083 443 A2, entitled ULTRASONIC IM- 
AGE APPARATUS FOR SEPARATING OBJECT, 
wherein each of internal organs of the human body to 

20 be diagnosed is separately extracted and visualized in 
athree-dimensional ultrasound image. In addition to the 
automatic extraction capabilities, the section for extract- 
ing a target object 1 06 can extract a target object drawn 
by the user. The extraction of a boundary of a drawn 

25 target object may be performed by either a three-dimen- 
sional segmentation based on the three-dimensional 
technique or by continuously segmenting the two-di- 
mensional volume data and manipulating them three di- 
mensionally. 

30 [0013] Figs. 3a and 3b show ultrasound images to ex- 
plain a procedure for extracting a target object from 
three-dimensional ultrasound image data acquired in 
accordance with the present invention. Fig. 3a illustrates 
an image resulting from three dimensionally rendering 

35 the boundary of the target object as drawn by the user 
in a xyz plane. The rendered image of the target object 
with its boundary contoured as shown in Fig. 3 can be 
advantageously used when an automatic extraction of 
the boundary of the target object is not feasible as in 

40 most of the ultrasound images or when the ultrasound 
imaging system being used is not equipped with such 
automatic extraction capabilities. Fig. 3b shows an im- 
age where the boundary of the target object has been 
extracted. After extracting a two-dimensional boundary 

45 of the target object in a xyz plane, a three dimensionally 
rendered image can be displayed. 
[0014] As mentioned above, section for extracting a 
target object 106 in accordance with the present inven- 
tion performs processing on continuous volume data, 

50 and thus can extract and display the target object by per- 
forming motion estimation instead of performing real- 
time continuous segmentation. Thus, section for ex- 
tracting a target object 1 06 acquires motion information 
of the target object in motion and provides location in- 

55 formation helpful in extracting the next target object. As- 
suming that there is a target object previously extracted 
by the first extracting step during the process of contin- 
uously extracting the target object, the entire or some 
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sample blocks in the previously extracted target object 
are pattern -matched with the neighboring areas of the 
subsequently extracted target object. Through this proc- 
ess, the final location of the target object can be calcu- 
lated, and the calculation indicates that the target object 
has moved by the calculated distance. Therefore, it is 
possible to finely and automatically modify the contour 
extracted from the previous volume based on the loca- 
tion information. 

[0015] Location-calculating section 108 calculates a 
center of gravity of the target object based on a geomet- 
ric shape of the extracted target object, and obtains in- 
formation on the location of the biopsy needle as repre- 
sented as a three-dimensional shape in the spatial co- 
ordinate system. The reason why the center of gravity 
of the target object is calculated is that the biopsy needle 
must be inserted into the center areaof the target object, 
and calculating a center of gravity is known as the most 
widely used method for finding a center of a geometric 
object. Based on the calculated center of gravity of the 
target object, location-calculating section 108 can thus 
estimate a motion of the target object currently in mo- 
tion, and allows guiding apparatus 100 to estimate the 
direction of advance of the biopsy needle. 
[001 6] Display section 1 1 0 displays the extracted tar- 
get object based on the well-known computer graphics 
techniques. Display section 1 1 0 may display only the ex- 
tracted target object in isolation, and may alternatively 
display only the boundary of the target object against 
the background. 

[0017] Controller 112 controls display section 110 to 
display a motion of the target object, the path of the bi- 
opsy needle currently being guided as calculated, and 
the ideal path between the target object and the biopsy 
needle whose information is useful for accurately guid- 
ing the biopsy needle. Controller 112 further controls 
display section 110 to display numerical information on 
an error between the ideal path and the actual path of 
the biopsy needle, together with graphics. Provided 
guide lines have a cylindrical shape with a fixed diame- 
ter. When the biopsy guide line is provided, the error is 
indicative of how the biopsy needle is parallel with the 
biopsy guide line or how closely the biopsy needle 
moves along the center of the biopsy guide line. With 
the numerical information on the error, an angle be- 
tween the biopsy needle and the biopsy guide line, and 
a distance between the center of the biopsy guide line 
and the tip of the biopsy needle can be defined as useful 
parameters. 

[0018] Referring now to Figs. 4a and 4b, ultrasound 
images are shown which explain a procedure for guiding 
a biopsy needle by extracting a target object in accord- 
ance with the present invention. In particular, Fig. 4a il- 
lustrates the boundary of the target object extracted on 
the xyz plane, and indicates how the biopsy needle is 
located in the three-dimensional image rendered with 
the use of information on the boundary. Fig. 4b illus- 
trates a process where the target object or the biopsy 



needle is tracked by exploiting the automatic tracking 
capabilities. 

[001 9] Controller 1 1 2 further controls guiding appara- 
tus 100 by using information on the center of gravity of 

5 the extracted target object, which is calculated in loca- 
tion-calculating section 108, and information on the lo- 
cation of the biopsy needle. First, controller 112 deter- 
mines whether the direction to which the biopsy needle 
is currently guided is correct by using information on the 

10 center of gravity and information on the location of the 
biopsy needle and, if an error is found, provides error 
information to location-calculating section 108 so that it 
can calculate and compensate the error. In this case, 
the error represents an error in terms of the biopsy nee- 

15 die's movement that is predicted to be generated as the 
biopsy needle is guided. The error is calculated based 
on information on the location of the extracted target ob- 
ject and information on the relative location of the biopsy 
needle. Next, controller 112 provides to section for ex- 

20 tracting a target object 1 06 location-related information 
that is helpful in sensing a motion of the target object 
and segmenting/extracting the next target object. 
[0020] Next, a procedure for observing a biopsy nee- 
dle in a three-dimensional ultrasound image provided in 

25 accordance with the present invention will be explained 
with reference to the flowchart shown in Fig. 5. The pro- 
cedure starts at step S502 where two-dimensional ul- 
trasound images are consequently acquired through ul- 
trasound transducer 1 02 and a volume image is gener- 
ic ated by three-dimensional image-forming section 104 
based on the acquired two-dimensional ultrasound im- 
age. In this case, once the biopsy guide line is set within 
the volume, the volume is rotated to be displayed as a 
straight line on the xy plane so that the biopsy guide line 

35 is displayed as a solid line on the xy palne. 

[0021] At step S504, the desired target object is ren- 
dered along the contour line which the user drew on the 
xyz plane image as acquired by section for extracting a 
target object 106, or otherwise the target object as au- 

40 tomatically segmented based on the VOCAL technique 
is designated. At step S506, the center of the extracted 
target object is found by location-calculating section 
1 08, and the volume display is set so that the target ob- 
ject is placed at the center of the volume display. Then, 

45 a figure indicative of the target object is displayed at the 
center of the xy plane image, as shown in Figs. 3a and 
3b, and the biopsy guide line is displayed to extend up 
to the center of the target object (i.e., the center of the 
plane). On the z plane, on the other hand, the biopsy 

50 guide line is displayed as a point on the center of the 
figure of the target object. 

[0022] At step S508, the xyz plane image and ren- 
dered volume display are aligned. For the free-hand 
style method, since information on the geometric struc- 
55 ture of the biopsy needle is not available, it is not pos- 
sible to display the predicted biopsy guide line. In this 
case, the VOCAL technique is applied to segment the 
biopsy needle, and the location of the biopsy needle is 
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found within the volume. Based on the location of the 
biopsy needle as found, displays relating to the biopsy 
needle are aligned on the xyz plane and rendered vol- 
ume display. 

[0023] At step S510, in order to display the guiding 
status of the biopsy needle, the rendered volume image 
as displayed together with the xyz plane image is rotated 
by a predetermined angle to the counter-clockwise di- 
rection and in turn to the clockwise direction. The reason 
for rotating the rendered volume image is that an image 
rendered from extracting the two or three-dimensional 
boundary or an image obtained based on the volume 
rendering method is displayed on the two-dimensional 
display screen, and therefore, the viewer may not feel 
the cubic effect if he continues to see such image. In 
this display mode, since the rendered image, which oth- 
erwise may become a 2.5 dimensional image, is dis- 
played more like a three-dimensional image, the oper- 
ator can observe the biopsy needle moving in the space. 
[0024] At step S51 2, the biopsy needle is continuous- 
ly tracked by using the live VOCAL technique. At the 
same time, the target object is also tracked so that a 
relationship between the biopsy needle and the target 
object can be clearly recognized on the xyz plane and 
volume image. More specifically, an image like that ob- 
tained when an endoscope camera is attached to the 
biopsy needle is added to the xyx plane image. Then, 
the user can visually appreciate the motion of the biopsy 
needle that approaches toward the target object. The 
target object is seen perspectively in the rendered status 
as if it is zoomed-in. As a result, the tissue that is found 
by the VOCAL technique based on the location of the 
tip of the biopsy needle can be seen as if an onion shell 
is peeled. 

[0025] At step S514, an alarm is generated as the bi- 
opsy needle approaches the target object. Once the bi- 
opsy needle approaches within a predetermined dis- 
tance from the target object, an alarm having a long in- 
termittence period and low frequency is first generated. 
As the biopsy needle further approaches the target ob- 
ject, an alarm having a relatively short intermittence pe- 
riod and high frequency is generated to alert that the 
biopsy needle is approaching near the target object. 
When the tip of the biopsy needle reaches the center of 
the target object, a continuous alarm is generated. Sim- 
ilarly, the color for the background of the z plane in the 
xyz plane image progressively turns into a brighter color, 
when the biopsy needle approaches the target object. 
This can give a visual alarm effect. 
[0026] For either case when the free-hand style meth- 
od or the method using the biopsy guide is used, the 
biopsy needle cannot be aligned on any of the x-y-z 
planes under certain situations. In this case, the biopsy 
needle is first found from the volume, and then geomet- 
ric information on the biopsy needle is extracted. There- 
after, the aligned xyz plane image and volume image 
are formed, in the manner as mentioned above. 
[0027] If the biopsy needle is used in an image mode 



other than the aligned xyz plane image mode or the vol- 
ume image mode, the biopsy guide line having perspec- 
tive is graphically displayed with respect to a point where 
the biopsy needle is currently passing. If the biopsy nee- 

5 die is tracked, such biopsy guide line is continuously dis- 
played in the xyz plane, and the operator may change 
the angle of view as the biopsy needle is guided. 
[0028] In accordance with the present invention, the 
biopsy needle can be aligned and displayed on the xyz 

10 plane. Also, a motion of the biopsy needle or the target 
object can be tracked so that a relationship between the 
biopsy needle and the target object can be clearly ob- 
served. Therefore, it is possible to enable the biopsy 
needle to exactly reach the target object. The prior art 

15 problem that the accuracy of diagnosis may depend on 
the skill of the user can also be eliminated. 
[0029] While particular embodiments of the present 
invention have been shown and described, it will be ob- 
vious to those skilled in the art that changes and modi- 

20 fications may be made without departing from this in- 
vention in its broader aspects and, therefore, the aim in 
the appended claims is to cover all such changes and 
modifications, as fall within the true scope of this inven- 
tion. 

25 

Claims 

1 . An apparatus for enabling a biopsy needle to be ob- 
30 served in a three-dimensional ultrasound image 

comprising: 

means for transmitting ultrasound signals to- 
wards a target object and receiving echo sig- 

35 nals reflected from the target object; 

means for forming a three-dimensional volume 
image by receiving and combining two-dimen- 
sional data from said transmitting means; 
means for extracting the target object from the 

40 three-dimensional volume image; 

means for estimating a location of the extracted 
target object and a location of the biopsy needle 
that is inserted into the extracted target object; 
means for displaying the extracted target ob- 

45 ject; and 

means for calculating an error in location be- 
tween the biopsy needle and the extracted tar- 
get object based on said estimation and provid- 
ing the calculated error to said estimating 

50 means. 

2. The apparatus according to claim 1, wherein said 
means for extracting extracts the target object 
based on one of a continuous segmentation and ex- 

55 traction process, and a motion estimation process. 

3. The apparatus according to claim 1, wherein said 
means for estimating finds a center of gravity of the 
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extracted target object to estimate the location of 
the extracted target object. 

4. The apparatus according to claim 1, wherein said 
means for displaying displays a path between the 5 
extracted target object and the biopsy needle that 

is to be followed by the biopsy needle. 

5. The apparatus according to claim 1 , wherein the er- 
ror is an error in terms of location of the biopsy nee- 10 
die to be generated as the biopsy needle is guided 
toward the extracted target object. 

6. A method of enabling a biopsy needle to be ob- 
served in a three-dimensional ultrasound image, 15 
comprising the steps of: 

acquiring a two-dimensional ultrasound image 
of a subject; 

generating a three-dimensional volume image 20 
based on the two-dimensional ultrasound im- 
age; 

segmenting a target image, which corresponds 
to a target object within the subject, from the 
three-dimensional volume image; 25 
displaying a guide line of the biopsy guide on 
the segmented target image; 
extracting the segmented target image; 
acquiring information on a location of the biopsy 
needle by reference to the segmented target 30 
image; 

calculating an error based on the information; 
and 

displaying a guiding status of the biopsy needle 
based on the calculated error. 35 



object. 

11. The method according to claim 6, wherein said ex- 
tracting step includes calculating a center of gravity 
of the segmented target image. 



7. The method according to claim 6, wherein the two- 
dimensional ultrasound image includes images on 
x, y and z planes of a rectangular coordinate sys- 
tem. 40 



8. The method according to claim 7, wherein the two- 
dimensional ultrasound image is aligned with re- 
spect to the biopsy needle to form the three-dimen- 
sional volume image. 45 

9. The method according to claim 8, wherein the three- 
dimensional volume image is generated through a 
rendering process and based on the two-dimen- 
sional ultrasound image. 50 

10. The method according to claim 6, wherein the error 
is an error in terms of location of the biopsy needle 
within the segmented target image that is to be gen- 
erated as the biopsy needle is guided toward the 55 
target object, said error being calculated with refer- 
ence to the information on a location of the biopsy 
needle and information on a location of the target 
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